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1  Introduction 


Sensors  are  ubiquitous  in  modern  society,  both  because  of  the  increased  need  for  surveillance 
and  information  for  real-time  decisions  and  because  of  the  decreased  cost  of  computation, 
communication,  and  data  storage.  Research  and  development  of  sensors  involves  interdisci¬ 
plinary  studies  with  hirelings  published  in  scientific  journals  such  as  IEEE  Sensors  Journal, 
International  Journal  of  Distributed  Sensor  Networks,  Ad  Hoc  &  Sensor  Wireless  Networks, 
and  International  Journal  of  Remote  Sensing. 

Sensors  have  found  wide  application  in  both  industrial  (see  Mandroli  et  al.  (2006)  and  the 
references  therein)  and  defense  contexts  (see  Rice  and  Willstatter  (2000),  Yost  and  Wash¬ 
burn  (2000)  and  Miller  (2007)).  This  paper  is  concerned  with  defense  applications.  Also, 
among  various  types  of  sensors  such  as  acoustic,  chemical,  biological  and  visual  sensors,  we 
are  interested  in  visual  sensors  which  may  or  may  not  possess  an  on-board  image  process¬ 
ing  capability.  We  are  particularly  interested  in  the  use  of  satellites  and  video  cameras  in 
surveillance  missions.  We  also  note  that  surveillance  is  required  in  both  day-  and  night-time. 
Therefore,  it  is  natural  to  assume  that  video  cameras  used  for  surveillance  possess  imaging 
infrared  (HR)  capabilities. 

Satellites  can  be  equipped  with  advanced  cameras  (Aplin  et  al.  1997),  and  have  recently 
been  used  to  observe  earth  for  events  such  as  forest  fires  and  volcanic  eruptions  (Damiani 
et  al.  2005).  Generally  observation  tasks  assigned  to  satellites  are  non- repetitive  and  op¬ 
timal  operation  of  a  satellite  is  achieved  by  maximizing  total  benefit  from  the  performed 
observations  (Pemberton  and  Galiber  2001,  Vasquez  and  Hao  2001,  Lemaitre  et  al.  2002). 
To  make  an  observation  a  satellite  has  to  focus  on  a  site  of  interest.  Therefore,  between  two 
consecutive  observations,  the  satellite  will  require  some  time  to  focus  on  the  new  site. 

Surveillance  using  video  cameras  has  improved  in  recent  years  (see  Hampapur  et  al. 
(2003)  and  Bagdanov  et  al.  (2005)  for  examples).  A  single  camera  or  a  network  of  cameras 
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can  be  used  to  simply  monitor  a  spot  or  track  a  moving  object  in  a  region  of  interest.  The 
shape  of  a  region,  and  the  number  and  sizes  of  obstacles  in  the  region  can  prohibit  effective 
surveillance  of  the  region  using  only  one  camera.  In  that  case  a  network  of  cameras  is  needed. 
A  distributed  approach  to  manage  the  network  is  to  equip  each  camera  with  an  on-board 
image  processing  capability.  A  centralized  approach  is  to  use  only  one  image  processor  which 
can  be  a  human  expert  or  an  automated  image  processing  algorithm. 

In  all  the  above-mentioned  papers,  the  goal  is  to  perform  non-recurring  tasks  such  as 
taking  a  snapshot  of  a  region  or  tracking  a  moving  object.  This  paper  is  concerned  with 
a  single  image  processor  used  to  monitor  multiple  sites  of  interest  in  a  region,  via  periodic 
observations.  For  convenience,  in  the  rest  of  the  paper,  we  refer  to  the  sensing  and  image 
processing  system  as  the  sensor  and  the  location  of  interest  as  the  site.  Although  the 
sensor  may  stay  at  a  single  physical  location  throughout  a  surveillance  mission,  processing 
images  from  geographically  separated  sites  one  at  a  time  can  be  considered  equivalent  to 
visiting  those  sites  in  some  order.  Hence,  we  refer  to  processing  an  image  from  a  site  as  a 
“visit”  by  our  “mobile  sensor”  to  that  site.  We  note  that  many  real-life  scenarios  can  be 
described  to  motivate  the  problem  addressed  in  this  paper;  we  present  only  two  due  to  space 
considerations. 

A  geo-synchronous  satellite  can  be  used  to  monitor  intrusions  along  a  border.  Sites  of 
interest  can  be  unprotected  points  or  paths  along  the  border  that  can  be  used  by  intruders. 
Obviously,  dedicating  a  fixed  sensor  (a  dedicated  satellite)  to  each  site  would  be  very  costly, 
whereas  a  single  sensor  can  be  used  to  efficiently  monitor  the  entire  region  by  observing  the 
sites  in  a  sequence.  Another  possible  application  of  this  type  is  to  monitor  uninhabited  areas 
such  as  sea  lanes,  woods  or  abandoned  facilities  for  suspicious  activities. 

In  our  second  example  we  consider  improvised  explosive  devices  (IEDs)  and  quote  a 
recent  study  by  White  (2007): 
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In  the  first  six  months  of  2006,  there  were  more  than  11,000  IED  attacks  in  Iraq, 
compared  to  5,607  in  2004  and  10,953  in  2005,  according  to  the  Iraq  Coalition 
Casualty  Count.  The  US-based  Homeland  Security  Research’s  ‘Global  Counter- 
IED  Markets  and  Technologies  Forecast  2008  to  2012’  reported  that  insurgents 
plant  IEDs  under  roads  and  gravel  in  order  to  avoid  bomb  sensors,  requiring  field 
commanders  to  use  a  variety  of  countermeasures,  including  increased  patrols  and 
closed-circuit  television. 

The  streets  of  a  city  connect  residential  and  business  neighborhoods  and  may  have  varying 
traffic  flows.  Some  locations  can  be  critical  targets  where  camera  towers  might  be  placed  to 
monitor  the  city.  Processing  images  from  one  camera  at  a  time,  instead  of  equipping  each 
camera  with  an  image  processing  capability,  can  be  a  practical  and  efficient  solution.  In 
this  scenario,  image  processing  is  performed  to  rapidly  yield  a  “yes”  or  “no”  answer  to  such 
questions  as  “is  anyone  entering  or  leaving  a  building?,”  “is  there  a  car  parked  near  a  critical 
location?”  or  “is  there  human  activity  adjacent  to  a  road?” 

As  discussed  earlier,  the  image  processor  in  these  examples  can  be  considered  a  mobile 
sensor.  At  this  point,  we  also  distinguish  our  work  from  that  on  fixed  sensors.  Challenges 
arising  from  the  use  of  fixed  sensors  include  determining  which  sensor  types  to  use,  the 
quantity  and  location  of  each  sensor  type,  and  how  to  communicate  in  a  network  of  sensors. 
With  a  mobile  sensor,  on  the  other  hand,  our  focus  is  on  scheduling  visits  to  the  sites. 

Tiwari  et  al.  (2005)  consider  a  single,  mobile  sensor  and  apply  a  result  from  control  and 
communication  theory  (Sinopoli  et  al.  2004)  to  quantify  when  a  single  sensor  is  sufficient 
to  maintain  a  reliable  estimate  of  linearly  evolving  parameters  in  multiple  areas  of  interest. 
Building  on  the  feasibility  criterion  for  a  single  sensor,  Yerrick  et  al.  (2007)  develop  stochastic 
sensor  scheduling  algorithms  for  this  same  problem.  That  is,  they  first  obtain  stationary 
probabilities  of  the  sensor  being  at  each  site  such  that  error  estimates  for  all  the  sites  remain 
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bounded.  They  then  either  use  the  stationary  probabilities  as  the  basis  for  a  probabilistic 
sensor  motion  strategy  or  obtain  a  matrix  of  transition  probabilities  in  a  Markov-chain 
setting.  Yavuz  and  Jeffcoat  (2007)  apply  results  from  the  deterministic  scheduling  literature 
to  develop  a  deterministic  sensor  scheduling  policy  to  minimize  the  cost  of  lost  information. 
In  all  these  papers,  the  activity  rates  of  the  sites  are  assumed  time-invariant  over  some 
planning  horizon.  In  this  paper,  we  generalize  the  results  of  Yavuz  and  Jeffcoat  (2007) 
to  allow  the  rate  of  activity  or  the  value  of  information  from  areas  under  surveillance  to 
evolve  as  a  function  of  time.  We  also  design  a  computational  experiment,  propose  both 
deterministic  and  probabilistic  scheduling  methods  and  present  a  computational  comparison 
of  the  proposed  methods. 

The  remainder  of  this  paper  is  organized  as  follows.  In  Section  2  we  introduce  a  mathe¬ 
matical  model  and  construct  a  computational  study.  In  Section  3  we  propose  both  determin¬ 
istic  and  stochastic  sensor  scheduling  methods,  as  well  as  a  hybrid  of  the  two  approaches. 
We  also  discuss  exact  optimization  approaches  and  lower  bound  policies  for  the  problem. 
Section  4  presents  the  results  from  our  computational  study,  and,  finally,  Section  5  provides 
a  summary  and  conclusions. 

2  Problem  definition 

In  this  paper,  we  assume  that  the  sensor  requires  a  constant  amount  of  time  to  visit  a  site 
and  make  an  observation.  For  a  geo- synchronous  satellite  this  implies  a  constant  effort  to 
re-focus  on  a  new  site.  For  the  image  processing  function,  it  is  valid  to  assume  a  constant 
image  resolution,  thereby  a  constant  time  to  process  the  image.  For  video  cameras,  the 
central  image  processor  can  instantly  select  the  next  camera  (equivalently,  the  next  image 
to  process).  Again,  a  common  image  resolution  implies  a  constant  image  processing  time, 
thereby  justifying  the  assumption.  Therefore,  we  build  our  mathematical  formulation  in 
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discrete-time  where  the  time  required  to  visit  a  site  and  process  the  image  is  taken  as  the 
“time-unit.” 

Let  Xitt  be  the  binary  decision  variable  denoting  whether  the  sensor  is  scheduled  to  visit 
site  i  at  time  t,  and  y%j,  denote  the  last  time  site  i  was  visited  as  of  the  end  of  time  t.  Note 
that  Hi)t  =  t  only  in  time  intervals  in  which  the  sensor  visits  site  i.  When  the  sensor  is  focused 
on  site  i,  it  updates  the  status  of  the  site.  In  other  words,  we  have  perfect  information  about 
the  site  in  that  time  step.  Since  the  sensor  cannot  focus  on  more  than  one  site  at  the  same 
time,  focusing  on  one  site  means  losing  information  about  the  current  states  of  the  other 
sites.  The  extent  of  the  information  loss  depends  on  the  activity  rate  of  a  site.  We  can 
afford  to  ignore  less  active  sites  for  a  large  number  of  time  steps,  whereas  more  active  sites 
must  be  visited  frequently.  In  this  paper,  we  assume  that  there  is  no  cost  for  movement  or 
observation;  our  whole  concern  is  the  information  loss. 

In  the  time-invariant  version  of  the  problem  we  associate  with  each  site  i  a  fixed  penalty 
a,-  and  a  variable  penalty  bi  of  information  loss.  More  specifically,  a  fixed  penalty  of  not 
visiting  a  certain  site  is  incurred  for  each  time  step  in  which  the  sensor  is  away  from  the 
site.  In  addition,  a  variable  penalty  is  incurred  for  each  time  unit  that  has  passed  since  the 
sensor’s  last  visit  to  that  site,  providing  ever-increasing  motivation  for  the  sensor  to  return 
to  a  neglected  site.  Determining  the  values  of  the  parameters  is  straightforward  for  many 
applications.  Recall  the  city  surveillance  example.  The  variable  penalty  bi  of  a  site  %  might  be 
the  average  number  of  people  or  vehicles  passing  through  the  site  in  a  time-unit.  Multiplying 
this  by  the  number  of  time  units  since  our  last  visit  to  this  site  yields  the  expected  number 
of  people  or  vehicles  that  have  passed  through  the  site  while  we  were  not  watching.  As  it  is 
reasonable  to  set  the  probability  of  having  an  incident  proportional  to  the  traffic  count,  this 
count  can  be  appropriately  scaled  to  estimate  the  probability  of  missing  an  incident  while  not 
watching  the  site.  The  fixed  penalty  a;  can  then  be  used  to  incorporate  long-term  historical 
data  into  the  model.  If  a  site  i  has  been  involved  in  many  incidents,  a  large  fixed  penalty  is 
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associated  with  it,  thereby  encouraging  the  sensor  to  visit  that  site  more  frequently  even  if 
the  rate  of  activity  at  the  site  is  low. 

We  introduce  time-variant  site  dynamics  to  the  model  by  allowing  bht  to  vary  throughout 
the  planning  horizon.  An  increase  in  the  variable  penalty  parameter  (i.e. ,  —  bl:t- 1  >  0  for 
some  t)  means  that  site  i  is  becoming  more  active  and,  hence,  requires  more  attention  from 
the  sensor.  In  the  city  surveillance  example,  it  is  natural  to  observe  increased  activity  rates 
in  rush  hours.  Similarly,  a  decrease  (i.e.,  bl:t  —  <  0  for  some  t)  means  that  the  activity 

level  of  site  i  is  diminishing,  so  that  the  sensor  may  visit  it  less  frequently.  Since  the  fixed 
penalty  parameter  a*  represents  long-term  history,  we  do  not  make  a*  time-variant  in  our 
study. 

The  objective  function  in  our  model  minimizes  the  maximum  penalty  incurred  for  a  sensor 
schedule  defined  over  a  finite  time  horizon.  This  is  achieved  by  defining  C  =  ma Xi)t{ai  + 
bitt{t  —  Minimizing  the  maximum,  known  as  the  min-max  approach,  allows  a  fair 

allocation  of  the  sensor’s  attention  to  sites.  Based  upon  a  known  planning  horizon  length 
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(T),  we  formulate  the  following  integer  linear  program. 


Minimize  C 

(1) 

Subject  to 

C  +  diXij  +  bijtyiit  >  a*  + 

Vi  =  1, 

..,n;Vt  =  1, T 

(2) 

n 

5>,t  =  i, 

Vt  =  1, 

T 

(3) 

i= 1 

Vi  ,t  Vi,t—  1  —  tXit, 

Vi  =  1, 

n;  Vt  =  1, ..,  T 

(4) 

Vi,t  — 

Vi  =  1, 

..,n]Wt  =  1, ..,  T 

(5) 

CD 

<3 

Vi  =  1, 

~,n 

(6) 

C  >  0, 

(7) 

Xi,t  e  {0,1}, 

Vi  =  1, 

■■ ,n-yt  =  1, ..,  T 

(8) 

Vi,t  e  {0}  u  z+, 

Vi  =  1, 

•• ,n-yt  =  1, ..,  T 

(9) 

The  model  is  built  as  a  fully  linear  model,  that  is,  the  objective  function  and  constraints 
are  all  linear  functions  of  the  decision  variables.  Note  that  defining  C  as  a  variable  and 
including  it  in  a  constraint  is  critical  for  the  linearity  of  the  formulation.  The  objective 
function  of  the  model  (1)  simply  aims  to  minimize  the  maximum  penalty  defined  by  the  first 
constraint  (2).  More  specifically,  C  >  a;(l  —  x^t)  +  &*,*(£  —  Vi,t),  for  all  i  and  t.  Constraint  (3) 
assures  that  the  sensor  visits  exactly  one  site  in  each  stage.  Constraints  (4)  and  (5)  together 
assure  that  ylji  is  updated  only  when  the  sensor  is  at  site  i,  and  remains  constant  at  other 
times.  Constraint  (6)  initializes  variable  y.  Finally,  constraints  (7-9)  define  the  decision 
variables  C,  x  and  y  as  nonnegative,  binary  and  nonnegative-integer,  respectively.  Also  note 
that,  if  all  the  input  parameters  a*  and  bitt  are  integers,  the  optimal  objective  function  value 
(C*)  will  be  integer,  as  well. 


In  this  paper,  we  construct  five  test  instances  with  which  we  can  test  and  compare  our 
solution  approaches.  Each  instance  consists  of  n  =  5  sites  and  a  planning  horizon  of  T  —  500 
time  units.  The  first  instance  is  time- invariant.  The  purpose  of  including  this  instance  is  to 
compare  the  performance  of  deterministic  and  stochastic  approaches,  a  comparison  that  has 
not  been  addressed  in  the  existing  literature  on  the  time-invariant  version  of  the  problem. 
The  second  and  third  instances  represent  monotonically  changing  site-dynamics.  The  rate 
of  change  in  site-dynamics  are  slow  and  fast  in  these  two  instances,  respectively.  In  both 
instances,  the  first  two  sites  become  more  active  with  time,  whereas  the  last  two  become  less 
active.  The  last  two  instances  represent  a  cyclic  pattern  in  the  site  dynamics.  In  the  former, 
site  dynamics  are  positively  correlated.  That  is,  all  sites  become  more  or  less  active  at  the 
same  time.  In  the  latter,  the  first  three  sites  and  the  last  two  sites  constitute  two  groups 
and  as  the  sites  in  the  first  group  become  more  active,  those  in  the  second  group  become 
less  active,  and  vice  versa.  The  structures  of  the  instances  are  presented  in  Tables  1  and  2. 
The  changes  of  the  monotonically  changing  instances  (in  Table  1)  are  given  with  a  two-field 
notation  (;)  where  the  numbers  before  the  are  the  periods  of  changes  and  the  number 
after  the  is  the  net  common  change  in  those  periods.  The  cyclic  changes  (in  Table  2)  are 
represented  with  a  more  compact  notation.  The  12  numbers  within  the  brackets  are  the  net 
changes  in  periods  that  are  multiples  of  ten.  More  specifically,  the  first  change  takes  place 
at  time  10,  next  one  at  time  20,  and  so  on.  The  first  cycle  is  completed  at  time  120.  We 
repeat  the  cycle  four  times,  thus  all  changes  are  completed  at  time  480. 

[Insert  Table  1  about  here.] 

[Insert  Table  2  about  here.] 
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3  Methods 


3.1  Deterministic  approach 

The  deterministic  approach  we  adopt  in  this  paper  is  the  well-known  greedy  method.  It 
starts  with  a  null  solution,  i.e.,  an  empty  sequence.  It  evaluates  all  n  sites  that  can  be 
visited  in  the  first  stage.  The  selection  of  the  next  site  to  visit  is  made  to  minimize  the 
penalty  of  information  loss,  i.e.,  penalty  of  not  visiting  a  site.  In  the  later  stages  the  method 
evaluates  n  —  1  sites  (all  but  the  one  just  visited)  for  its  next  visit.  These  simple  selection 
and  update  operations  are  repeated  until  a  complete  solution  (a  sequence  of  T  stages)  is 
obtained.  The  time  complexity  of  this  procedure  is  0{nT).  It  can  easily  be  shown  that  the 
method  does  not  necessarily  construct  optimal  sequences,  hence  is  a  heuristic.  We  refer  to 
this  simple  greedy  procedure  as  G. 

We  improve  the  performance  of  the  greedy  method  by  adding  a  look-ahead  feature.  The 
heuristic  still  evaluates  n  —  1  possible  sites  to  visit  in  each  stage,  but  makes  the  decision 
based  on  the  penalty  observed  in  the  next  l  stages.  The  computational  complexity  of  the 
enhanced  procedure  is  0{n2£T).  Larger  £  values  are  expected  to  yield  better  (lower  penalty) 
solutions  on  the  average,  although  this  is  not  guaranteed.  On  the  other  hand,  the  number 
of  operations  to  perform  increases  linearly  with  £,  thereby  rendering  small  i  values  more 
computationally  efficient.  In  our  preliminary  experiments  we  found  £  —  n  works  best  for 
most  problems.  We  call  this  procedure  greedy  with  look-ahead  and  refer  to  it  as  GLA. 

3.2  Stochastic  approach 

In  this  section  we  develop  a  general  stochastic  sensor  motion  framework.  The  framework  is 
built  on  the  idea  of  optimizing  steady  state  probabilities  of  the  sensor  to  be  located  at  each 
site.  We  also  define  optimal  visit  periods  for  the  sites  which  are  tied  to  the  steady  state 
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probabilities. 

Let  Ti  G  R  be  the  visit  period  to  site  i.  For  the  sake  of  simplicity,  we  assume  r*  does 
not  have  to  be  an  integer  and  any  r*  >  0  can  be  achieved  in  an  ideal  schedule.  Then,  the 
penalty  of  information  loss  at  site  i  is  a*  +  (r,;  —  at  time  t.  Here,  let  us  postpone  the 
discussion  of  the  time- variability  of  bijt  and  reduce  the  parameter  to  b%.  In  other  words,  let 
us  consider  a  sufficiently  small  planning  horizon  with  time-invariant  site  dynamics.  Visiting 
site  i  every  >  0  periods  is  equivalent  to  spending  7 q  =  l/r*  of  the  sensor’s  time  at  that  site. 
Obviously,  the  more  time  spent  at  a  site,  the  less  penalty  incurred  at  that  site.  Therefore, 
we  can  conclude  that  in  an  optimal  schedule  JV  7 q  =  1;  i.e. ,  all  the  available  time  is  utilized. 
We  also  know  that  in  reality  the  sensor  never  stays  at  any  site  for  two  consecutive  time  steps. 
Thus,  rt>  2  (equivalently,  7Tj  <  0.5)  should  be  satisfied  for  each  site  i.  Putting  all  together, 
we  formulate  the  following  non-linear  program  to  obtain  optimal  stationary  probabilities. 


Minimize  C  =  max  <  a,  +  f - 1  |  6, 1 

*  l  U  )  J 

(10) 

Subject  to 

n 

X^  =  1’ 

i= 1 

(11) 

7 Ti  <  0.5, 

V*  =  1,  ..,71 

(12) 

C  >  0, 

(13) 

7 Tj  G  M, 

\/i  =  1, n 

(14) 

This  non-linear  program  is  defined  on  continuous  variables  only  and  is  easily  solved. 
Utilizing  constraint  (12),  we  define  a  lower  bound  on  the  objective  function  value  with 
CL  =  maxj{aj  +  £>*}.  Then,  we  set  C  =  CL  and  calculate  =  [C  —  a.i)/bi  +  1  and  7 q  =  1/r* 
for  all  i.  Note  that  7 r*  =  0.5  for  the  sites  with  a*  +  fej  =  C  and  7 q  <  0.5  for  the  rest.  If 
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)>A  7T,;  =  1,  then  we  have  found  the  optimal  solution,  and,  hence  can  terminate.  If  <  1, 

we  again  have  found  the  optimal  objective  function  value.  In  order  to  satisfy  constraint 
(11),  we  arbitrarily  select  a  site  with  7 r*  <  0.5  and  increase  its  stationary  probability  by 
the  amount  needed  to  satisfy  the  constraint.  Note  that,  in  this  case,  in  the  initial  solution 
we  have  exactly  one  site  with  7q  =  0.5,  thus  we  can  find  a  feasible  solution  after  only  one 
step.  If  JA  7q  >  I;  on  the  other  hand,  the  current  C  value  is  not  feasible.  In  this  case,  any 
systematic  search  procedure  (such  as  bisection  search)  can  be  used  to  find  a  C  >  CL  value 
that  makes  ^A  7q  sufficiently  close  to  1. 

Example  1.  Consider  the  first  test  instance  in  Table  1  with  (a*,  bj)  pairs  (125,25),  (155,15), 
(170,  10),  (165,  5)  and  (95,  15)  for  the  five  sites,  respectively.  CL  =  max{150, 170, 180, 170, 110} 
180,  and  it  yields  =  0.313  +  0.375  +  0.500  +  0.250  +  0.150  =  1.588,  in  which  case 

we  have  to  search  for  a  C  >  CL.  A  candidate  is  C  =  CL  x  1.588  =  285.75,  yielding 

JA  7 Ti  =  0.135  +  0.103  +  0.080  +  0.040  +  0.073  =  0.430.  Consequently,  we  have  to  search  for 
a  C  such  that  180  <  C  <  285.75.  C  =  200  yields  exactly  JA  7q  =  1  for  this  instance. 

The  above  approach  is  developed  for  time-invariant  site  dynamics.  Therefore,  when  the 
site  dynamics  change,  the  transition  probabilities  have  to  be  re-calculated  to  reflect  the 
changes. 

Example  2.  Consider  the  second  test  instance  in  Table  1  with  (a,i,bi)  pairs  equal  to 

(125,25)  for  all  five  sites,  and  CL  =  150  initially.  The  optimal  solution  can  be  easily  calcu¬ 

lated  as  C  =  225  and  7 q  =  0.200  for  all  sites.  At  time  20,  the  variable  penalty  of  the  first  site 
increases  to  30  units  thereby  calling  for  re-calculating  the  optimal  stationary  probabilities. 
The  solution  to  this  new  sub-problem  is  C  =  228.83  which  is  achieved  by  setting  7Ti  =  0.224 
and  7 Ti  =  0.194  for  all  other  sites.  As  seen  from  this  example,  the  site  with  the  increased 
variable  penalty  receives  more  sensor  attention  at  the  expense  of  the  other  sites. 
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Up  to  this  point,  we  have  discussed  a  simple  stochastic  sensor  motion  policy  based  on 
optimal  stationary  probabilities  of  the  sensor  being  at  a  site.  This  simple  policy  is  static. 
That  is,  the  transition  probabilities  are  constant  over  time.  However,  since  the  objective 
function  is  the  minimization  of  maximum  penalty,  a  site  becomes  more  critical  as  the  number 
of  time  steps  away  from  it  increases.  Therefore,  reducing  the  probability  of  visiting  a  site 
that  has  been  visited  recently  and  increasing  the  probability  of  visiting  a  site  that  is  overdue 
can  improve  the  quality  of  the  solutions  obtained.  We  elaborate  on  this  point  in  the  following 
discussion. 

Recall  that  y%x  is  an  integer  decision  variable  representing  the  last  time  site  i  was  visited 
by  the  end  of  time  t.  When  constructing  a  visit  sequence  one  starts  with  yij0  =  0  for  all  i. 
Furthermore,  in  the  first  stage  (t  —  1),  the  site  to  visit  is  not  known,  hence,  one  has  to  use 
Ui,i  =  Vi, o  =  0  for  all  i.  After  selecting  the  site  to  visit  in  the  first  stage  (say,  the  last  visit 
time  for  that  site  can  be  updated  (yq,i  =  1)-  Generalizing  this  to  any  stage  t,  we  state  that 
the  decision  is  made  using  yljt  =  y^t-x  for  all  i,  and  after  the  site  to  visit  is  selected  (i%)  the 
last  visit  time  for  that  site  is  updated  (yi*,t  —  t).  Now  we  focus  on  the  process  of  tuning  the 
visit  probabilities  for  each  site. 

Let  qi  be  a  preference  value  associated  with  visiting  site  i  in  a  given  stage  t.  (We  use 
qi  instead  of  q%x  for  convenience.)  As  discussed  earlier,  in  an  optimal  schedule  the  sensor 
never  stays  at  the  same  site  in  two  consecutive  time  steps.  Accordingly,  we  set  q^*  =  0. 

For  all  other  sites  we  calculate  the  visit  preference  using  qi  =  7q  x  ^  .  Here,  k  is  a 

user-defined  parameter  representing  the  weight  of  the  adjustment  factor  which  is 

smaller  than  1  for  the  sites  that  have  been  visited  within  their  ideal  (and  expected)  visit 
periods  and  greater  than  1  for  the  overdue  sites.  Using  a  larger  k  value  puts  more  emphasis 
on  visiting  the  overdue  sites.  Also  note  that  using  k  —  0  disables  the  adjustment  factor, 
or,  equivalently,  uses  the  stationary  probabilities  in  a  static  manner.  When  the  preference 
values  are  calculated  for  all  sites,  they  can  be  converted  to  probabilities  with  pt  =  qt/Q  for 


13 


all  i  where  Q  =  Y^i=i  Qi-  The  pseudo-code  of  the  stochastic  approach  is  presented  in  Figure 
1. 

[Insert  Figure  1  about  here.] 

3.3  A  hybrid  approach 

The  stochastic  approach  described  in  the  previous  section  is  based  on  optimized  stationary 
probabilities  of  the  sensor  being  focused  on  each  site.  Therefore,  after  the  optimization  of 
the  stationary  probabilities,  the  method  pays  almost  no  attention  to  the  objective  function 
contribution  of  the  decisions  made  in  each  stage  of  the  sequence.  The  greedy  method  de¬ 
scribed  earlier,  on  the  other  hand,  makes  the  decisions  solely  based  on  the  objective  function 
contribution.  In  this  section,  we  present  a  hybrid  of  the  two  approaches  which  can  also  be 
called  probabilistic  greedy  method. 

In  the  hybrid  method,  one  first  calculates  the  penalty  of  not  visiting  each  site  Cj  = 
ck  +  bi)t(t  —  y^t).  The  next  step  is  to  calculate  preference  values  to  visit  each  site  with 
a  =  ( -a-  )  .  Here,  cmax  =  maxHc;}  is  the  maximum  penalty  of  not  visiting  a  site.  Also, 

V  cmax  J  L  J 

the  implementation  is  similar  to  the  stochastic  method  as  q%*t_l  =  0  is  set  for  the  site  just 
visited,  and  Q  =  YhiQi  is  used  to  transform  the  visit  preferences  to  probabilities  (p*  =  qt/Q 
for  all  i ).  The  pseudo-code  of  the  hybrid  approach  is  presented  in  Figure  2. 

[Insert  Figure  2  about  here.] 

3.4  Exact  optimization  and  lower  bounds 

Yavuz  and  Jeffcoat  (2007)  show  that  the  time-invariant  version  of  the  problem  is  NP-Hard. 
Since  allowing  time-variant  site  dynamics  is  a  generalization  of  the  time-invariant  problem, 
the  time-variant  problem  studied  in  this  paper  is  NP-Hard,  as  well.  Therefore,  exact  opti¬ 
mization  approaches  are  likely  to  be  prohibitively  time-consuming.  To  test  this  hypothesis 
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we  used  a  commercially  available  optimization  software,  C-Plex  9.1,  and  observed  that  none 
of  the  five  test  instances  (in  Tables  1  and  2)  were  solved  within  30  minutes.  Therefore,  we 
focus  on  obtaining  lower  bounds  for  the  objective  function  values  of  the  five  test  instances 
addressed  in  this  paper. 

As  stated  earlier,  the  sensor  has  to  spend  at  least  one  time  unit  away  from  a  site  i,  thus 
incurring  a  cost  of  at  +  bi)t  at  that  site.  Therefore,  CL  =  max^a;  +  bi)t}  is  a  valid  lower 
bound  for  the  problem. 

ft  is  easy  to  see  that  in  many  real-life  instances  (especially  with  a  large  number  of  sites) 
the  sensor  has  to  spend  more  than  one  time-step  away  from  a  site.  Therefore,  CL  is  not 
necessarily  a  tight  lower  bound.  To  obtain  a  better  lower  bound,  we  focus  on  sub-problems 
and  then  solve  each  sub-problem  with  C-Plex  9.1.  In  our  experiments,  we  found  that  sub¬ 
problems  with  T  =  16  stages  were  easily  solvable.  Thus,  we  formulated  50  sub-problems  for 
each  instance  such  that  sub-problem  r  covers  the  stages  [(r  — 1)  x  10  +  1, ..,  min{(r  —  1)  x  10  + 
16,500}],  r  =  1,  ..,50.  Note  that  this  approach  does  not  provide  a  perfect  decomposition, 
since  the  sub-problems  are  overlapping.  The  objective  function  values  of  the  sub-problems 
are  compared  and  their  maximum  is  used  as  a  lower  bound  to  the  problem.  Since  the 
objective  is  of  the  min-max  type,  the  overlap  between  the  subproblems  does  not  inversely 
impact  the  lower  bound,  but  may  in  fact  strengthen  it. 

The  first  test-instance  is  time- invariant,  thus  all  the  sub-problems  are  identical.  In  fact, 
the  optimal  solution  to  this  instance  can  be  easily  constructed.  If  integer  periods  r,  to 
visit  sites  i  —  1,  ..,5  exist  such  that  a*  +  (r*  —  1  )bi  =  C  for  all  i  and  r*  are  achievable  in 
a  cyclic  schedule,  then  C  is  the  optimal  objective  function  value.  The  proof  is  based  on 
the  observation  that  if  one  site  is  given  more  time,  then  the  other  sites  must  receive  less 
time  and  their  penalties  must  be  larger.  Consequently,  the  maximum  of  the  penalties  is 
minimized  by  a  solution  that  makes  all  the  penalties  equal.  C  =  40  is  achievable  for  the 
first  instance  by  setting  rq  =  r2  =  r3  =  4  and  r4  =  r5  =  8,  and  repeating  the  visit  sequence 
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1  —  2  —  3  —  4  —  1  —  2  —  3  —  5. 


4  Results 

In  this  section  we  report  and  discuss  the  results  from  our  computational  study.  All  five- 
instances  are  first  divided  into  sub-problems  and  lower  bounds  are  obtained.  Then,  both 
the  simple  greedy  (G)  and  the  greedy  with  look-ahead  (GLA)  methods  are  implemented. 
The  stochastic  method  is  implemented  with  four  different  levels  of  the  input  parameter 
k  —  0,1,  2,  3.  Recall  that  k  =  0  is  equivalent  to  using  the  optimized  stationary  probabilities 
in  a  static  fashion.  Finally,  the  hybrid  method  is  implemented  with  three  different  levels 
of  the  input  parameter  k  —  1,2,  3.  All  together,  we  experiment  with  ten  methods:  LB,  G, 
GLA,  S(0),  S(l),  S(2),  S(3),  H(l),  H(2)  and  H(3). 

The  stochastic  and  hybrid  methods  construct  solutions  in  a  probabilistic  fashion,  thus 
we  run  them  100  times  on  each  instance  and  report  average-case  results. 

All  the  methods  take  negligible  time  on  a  desktop  computer  with  a  P4-3.4GHz  CPU 
and  2GB  of  memory.  Therefore,  we  do  not  report  on  the  computation  time  taken  by  the 
methods.  We  compare  the  methods  using  two  criteria,  namely  solution  quality  and  solution 
variability. 

Let  CLB  be  the  lower  bound  value  calculated  for  the  objective  function.  Also,  let  Cx  be 
the  objective  function  value  of  the  solution  obtained  by  some  method  X.  Then,  the  solution 
quality  for  method  X  is  calculated  as  x  100%,  so  that  smaller  values  represent  better 

solutions. 

Our  variability  criterion  is  adapted  from  Corominas  et  al.  (2007).  We  analyze  a  solution, 
i.e. ,  a  complete  sequence  of  500  stages,  and  note  the  distances  between  two  consecutive  visits 
to  each  site.  These  distances  are  stored  in  variables  77^,  i  —  1, . . ,  n  and  v  =  1, ..,  T$,  where 
T i  is  the  number  of  visits  to  site  i.  When  the  visit  intervals  are  determined,  their  average 
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(fji)  is  calculated.  Finally,  the  (per  stage)  variability  of  the  visit  intervals  is  calculated  with 
—Vi)2/T-  Again,  for  the  stochastic  and  hybrid  methods,  we  report  the  averages 
of  the  100  runs  performed  with  each  setting  of  the  method.  The  results  are  presented  in 
Table  3. 


[Insert  Table  3  about  here.] 

The  results  presented  in  Table  3  reveal  that  relative  deviation  from  the  lower  bound 
and  variability  are  positively  correlated.  Stated  another  way,  high  variability  tends  to  be 
associated  with  poor  solution  quality. 

One  quick  observation  from  the  table  is  the  extremely  high  variability  and  poor  solution 
quality  of  S(0)  compared  with  the  other  methods.  This  is  due  to  its  static  nature  in  using 
the  stationary  probabilities  to  construct  a  sequence. 

At  the  opposite  extreme  are  the  deterministic  methods  G  and  GLA.  They  both  yield  low 
variability  and  good  solution  quality.  When  we  compare  the  two  with  each  other,  we  observe 
that  GLA  consistently  yields  a  better  solution  quality  and  a  greater  variability. 

For  the  probabilistic  methods,  the  first  result  we  observe  is  that  as  k  increases  both 
methods  behave  much  like  the  greedy  method.  In  other  words,  smaller  k  values  represent  a 
higher  degree  of  randomness  and  larger  k  values  represent  an  almost  deterministic  selection 
of  the  next  site  to  visit.  When  we  compare  the  stochastic  and  the  hybrid  methods  with  each 
other,  we  observe  that  S (k)  yields  a  better  solution  quality  and  a  smaller  variability  than 
H(/c),  for  k  —  1,  2, 3. 

In  many  real-life  scenarios  a  high  variability  may  be  desired,  since  it  is  equivalent  to  un¬ 
predictability  of  the  schedule.  In  such  scenarios,  we  infer  two  results.  First,  GLA  dominates 
G  with  its  better  solution  quality  and  greater  variability.  Second,  GLA  and  the  probabilistic 
methods  are  all  appropriate  and  one  should  select  a  method  based  on  the  degree  of  variabil¬ 
ity  and  solution  quality  desired.  On  the  other  hand,  in  some  applications,  e.g.,  search  and 
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rescue  missions,  a  low  variability  is  desired.  Similar  scenarios  arise  in  the  fields  of  manu¬ 
facturing  and  supply  chain  management  (see  Corominas  et  al.  2007).  In  such  scenarios,  the 
results  obtained  in  this  paper  show  that  deterministic  methods  are  preferable  to  probabilistic 
methods. 


5  Conclusions 

In  this  paper  we  have  addressed  the  problem  of  scheduling  a  single  sensor  to  visit  a  number 
of  sites  with  possibly  time-variant  dynamics.  This  problem  is  an  extension  of  the  sensor 
scheduling  problem  that  has  been  studied  previously  with  only  time- invariant  site  dynamics. 
The  contributions  made  in  the  paper  include  the  development  of  both  deterministic  and 
stochastic  sensor  scheduling  methods  and  a  comparison  of  the  two  approaches. 

For  testing  purposes  we  created  five  test  instances  each  with  different  characteristics. 
The  results  show  that  the  methods  proposed  in  this  paper  are  all  time-effective.  When 
compared,  the  methods  show  a  trade-off  between  solution  quality  and  variability  of  the 
schedules  obtained.  Therefore,  the  selection  of  the  most  appropriate  method  should  be 
made  by  the  user  based  on  the  desired  solution  quality  and  variability. 
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Tabic  1:  Test  instances  1-3 


Instance 

i 

0 

bi 

Changes  in  a^t 

1 

1 

125 

25 

none 

2 

155 

15 

none 

3 

170 

10 

none 

4 

165 

5 

none 

5 

95 

15 

none 

2 

1 

125 

25 

(20,  140,  180,  300,  340;  +5) 

2 

125 

25 

(60,  120,  200,  260,  400;  +2) 

3 

125 

25 

none 

4 

125 

25 

(80,  160,  220,  320,  380;  -2) 

5 

125 

25 

(40,  100,  240,  280,  360;  -5) 

3 

1 

125 

25 

(60,  140,  220,  300,  380;  +5) 

2 

125 

25 

(25,  55,  85,  115,  145,  175,  205,  235,  265,  295,  325,  355;  +2) 
(370;  +1) 

3 

125 

25 

none 

4 

125 

25 

(30,  50,  70,  90,  110,  130,  150,  170,  190,  210,  230,  250;  -2) 
(260; -1) 

5 

125 

25 

(80,  160,  240,  320,  400;  -5) 

_ Table  2:  Test  instances  4-5 _ 

Instance  i  ait0  bi  Cyclic  changes  in  ai)t 

~ ~4  1  125  25  (+3,  +2,  +1,  -1,  -2, ’-3,  -3,  -2,  -1,  +1,  +2,  +3) 

2  100  20  (+3,  +2,  +1,  -1,  -2,  -3,  -3,  -2,  -1,  +1,  +2,  +3) 

3  150  25  (+3,  +2,  +1,  -1,  -2,  -3,  -3,  -2,  -1,  +1,  +2,  +3) 

4  175  15  (+6,  +4,  +2,  -2,  -4,  -6,  -6,  -4,  -2,  +2,  +4,  +6) 

5  125  30  (+3,  +2,  +1,  -1,  -2,  -3,  -3,  -2,  -1,  +1,  +2,  +3) 

5  1  125  25  (+3,  +2,  +1,  -1,  -2,  -3,  -3,  -2,  -1,  +1,  +2,  +3) 

2  100  20  (+3,  +2,  +1,  -1,  -2,  -3,  -3,  -2,  -1,  +1,  +2,  +3) 

3  150  25  (+3,  +2,  +1,  -1,  -2,  -3,  -3,  -2,  -1,  +1,  +2,  +3) 

4  175  15  (-6,  -4,  -2,  +2,  +4,  +6,  +6,  +4,  +2,  -2,  -4,  -6) 

5  125  30  (-3,  -2,  -1,  +1,  +2,  +3,  +3,  +2,  +1,  -1,  -2,  -3) 
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Figure  1:  Pseudo-code  for  Algorithm  Stochastic  (A;) 
Algorithm  Stochastic(fc) 

BEGIN 

1.  Set  I/,;. o  =  0,  i  =  1, 2, n  and  =  0. 

2.  FOR  t  =  1  to  T,  increase  t  by  1 
BEGIN 

3.  IF  site  dynamics  has  changed  THEN 

Obtain  7 u  and  r,  for  all  i  by  solving  the  NLP  in  page  6. 

4.  Set  Q  =  0. 

5.  FOR  i  =  1  to  n,  increase  i  by  1 
BEGIN 

6.  Set  yi<t  = 

7.  IF  i  =  *?_! 

THEN 

8.  Set  qi  =  0. 

ELSE 

9.  Set  qi  =  t u  • 

10.  Update  <2  <—  <2  +  Qi- 
END 

11.  FOR  7  =  1  to  n,  increase  i  by  1 
BEGIN 

12.  Set  pi  =  %. 

END 

13.  Select  i\  w.r.t.  probabilities  pt , *  =  l,..,n. 

14.  Set  yi*tt  =  t. 

END 

END. 


22 


Figure  2:  Pseudo-code  for  Algorithm  Hybrid (/c) 
Algorithm  Hybrid(fc) 

BEGIN 

1.  Set  yifl  =  0,  i  =  1, 2, n  and  =  0. 

2.  FOR  t  =  1  to  T,  increase  t  by  1 
BEGIN 

3.  Set  cmax  =  0. 

4.  FOR  i  =  1  to  n,  increase  i  by  1 
BEGIN 

5.  Set  yiit  =  yi't-i. 

6.  IF  i  =  i*_i 
THEN 

7.  Set  Ci  =  0. 

ELSE 

BEGIN 

8.  Set  Ci  =  ai  + 

9.  IF  cmax  <  Ci  THEN 

10.  Update  cmax  +  c^. 

END 

END 

9.  Set  Q  =  0. 

11.  FOR  i  =  1  to  n,  increase  i  by  1 
BEGIN 

12.  Set  qi  =  . 

13.  Update  <2  <—  <2  +  <?*■ 

END 

14.  FOR  i  =  1  to  n,  increase  i  by  1 
BEGIN 

15.  Set  Pi  =  %■ 

END 

16.  Select  w.r.t.  probabilities  pi,  i  =  1 

17.  Set  yi^t  =  t. 

END 

END. 
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